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Abstract 

This  study  is  an  initial  effort  to  measure  the  quality  of  nursing  care 
through  the  assessment  of  patient  outcome  rather  than  by  medical  care 
process.  The  researchers  generated  and  refined  539  measurement 
items,  utilizing  eight  universal  and  10  health  deviation  self-care  de- 
mand categories  adapted  from  work  by  D.  Orem  (1971).  Content  val- 
idity has  been  established  for  all  539  items.  Pretesting  and  inter- 
observer  reliability  testing  of  measurement  techniques  were  conducted 
for  414  of  the  items.  Development  of  an  instrument  for  matching 
measurement  techniques  to  patient  populations  was  initiated.  A  man- 
ual provides  detailed  instructions  on  the  use  of  the  instrument. 
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Foreword 


The  increasing  concern  in  public  policy  with  cost  containment  in  the 
health  care  delivery  system  in  no  way  obviates  the  continued  need  for 
quality  in  the  health  care  being  administered.  In  order  to  cut  costs  ef- 
fectively, information  is  needed  on  the  effects  on  quality  of  care  re- 
sulting from  changes  in  health  care  delivery  systems. 

Nursing,  the  major  contributor  in  the  delivery  of  care,  needs  measures 
by  which  to  evaluate  the  impact  of  changes  in  nursing  roles,  functions 
and  utilization  patterns.  This  research  study  supported  by  the  National 
Center  for  Health  Services  Research  presents  outcome  criteria  de- 
veloped as  part  of  a  new  approach  to  assessing  the  effectiveness  of 
nursing  care.  The  measures  to  assess  the  quality  of  care  are  based  upon 
eight  universal  and  ten  health  deviation  self-care  demand  categories. 
The  study  recognizes  that  outcomes,  by  definition,  are  the  result  of 
multiple  determinates.  Outcomes  are  the  combined  result  of  several 
converging  forces:  the  restorative  process  inherent  within  each  indi- 
vidual; environmental  and  social  influences;  and  the  contributions  of 
participative  health  professions.  The  study  also  recognizes  that  nurses 
and  nursing  care  contribute  to  outcomes  in  interaction  with  all  other 
health  professionals.  To  facilitate  interdisciplinary  assessment,  a  system 
is  described  which  matches  the  criterion  measures  of  nursing  care  to 
appropriate  diagnoses  and  Hospital  Index  (HICDA)  codes.  This  sys- 
tem will  permit  joint  assessments  involving  other  health  professionals, 
a  long-term  need  in  studies  of  this  kind. 

Gerald  Rosenthal,  Ph.D. 
Director 
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Introduction 


This  study  addresses  the  issue  of  evaluating  the  quality  of  nursing  care 
provided  adult,  medical-surgical,  hospitalized  patients.  The  increasing 
complexity  of  health  care,  changing  societal  demands  for  services,  and 
the  thrust  to  provide  quality  care  to  more  people  in  a  more  efficient 
and  more  accessible  manner,  necessitates  systematic  objective  evalua- 
tion of  the  delivery  of  health  care  services.  Nursing  constitutes  the 
largest  number  of  health  professionals  functioning  in  acute-care 
facilities  and  nurses  are  responsible  for  the  patient  continuously  from 
admission  to  discharge.  The  impact  of  nursing  care  services  must  be 
measured. 

Existing  evaluation  tools  in  nursing  reflect  measures  of  structure  and 
process.  Little  research  effort  has  been  devoted  to  the  development 
and  testing  of  outcome  measures.  It  is  recognized  that  patient  out- 
comes arise  from  multiple  determinations.  Outcomes  are  the  combined 
result  of  several  converging  forces:  the  restorative  process  inherent 
within  each  individual,  environment,  and  social  influences,  and  the 
contributions  of  all  participative  health  professionals  (i.e.,  nurses, 
physicians,  physical  therapists,  social  workers).  Nevertheless,  a  method 
to  assess  nursing's  impact  on  patients'  health  status  is  urgently  needed. 

The  objective  of  the  project  was  to  develop,  refine,  and  validate 
measures  of  the  quality  of  nursing  care.  The  focus  was  on  the  assess- 
ment of  the  outcomes  of  the  nursing  care  process  as  reflected  by  the 
patients'  health  status.  Measures  of  patients'  physical  and  emotional 
status,  and  the  extent  of  their  knowledge  and  ability  to  perform  self- 
care,  were  developed  and  related  to  universal  and  health  deviation 
demands.  Thereafter,  the  investigators  sought  to  ascertain  the  reliabil- 
ity and  validity  of  each  of  the  various  measurment  items  generated. 
Due  to  the  research  and  developmental  focus  of  the  work,  no  specific 
hypotheses  were  formulated  or  tested.  A  total  of  539  measurement 
items  were  developed  and  validated,  using  eight  universal  and  10 
health  deviation  self-care  demand  categories  developed  by  Orem  (1). 
In  addition,  414  of  the  items  were  pretested  to  determine  deviations  in 
findings  returned  by  different  observers.  This  subjective  variation  was 
corrected  by  the  development  of  specific  observation  procedures  for 
specific  patient  groups.  The  validated  measurement  items  and  an  "in- 
strument" for  matching  observer  techniques  to  particular  patients  were 
incorporated  in  a  manual  (2). 
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Analytical 

and  methodological 
procedures 


A  comprehensive  instrument  for  measuring  the  quality  of  nursing  care 
depends  on  specifying  the  domain  of  care  for  which  nursing  is  respon- 
sible. A  major  difficulty  from  the  beginning  was  that  nursing  did  not 
have  a  well-defined  classification  system  for  identifiable  patient  care 
problems.  Thus,  it  was  necessary  to  adopt  at  least  a  rudimentary  system 
for  organizing  and  definding  nursing  care  problems.  Three  ap- 
proaches were  explored;  a  discussion  of  each  approach  follows. 

Initially,  specific  medical  diagnoses  were  chosen  as  a  heuristic  device 
to  structure  our  thinking  about  nursing  care  problems.  Frequently  oc- 
curring medical  diagnoses  were  identified  for  medical-surgical  hos- 
pitalized patients.  Sources  included  Payne  (3),  the  Commission  of  Pro- 
fessional and  Hospital  Activities  (CPHA)  (4),  and  The  National  Center 
for  Health  Statistics  (NCHS)  (5).  For  each  medical  diagnosis  nursing 
care  problems  were  identified,  the  ameliorative  activities  for  each 
problem  were  listed,  and  the  desired  outcomes  for  those  activities  were 
described.  Finally,  those  desired  outcomes  were  translated  into  quanti- 
fiable measures,  usually  subsets  of  questions  to  ask  or  observations  to 
make  about  patients. 

It  soon  became  obvious  that  the  disease  entity  classification  was  too 
detailed  and  repetitious  to  be  appropriate  for  nursing.  That  is,  patients 
classified  differently  by  medical  diagnoses  were  often  similar  in  terms 
of  nursing  problems.  For  example,  the  paraplegic  rehabilitation  patient 
and  the  major  stroke  patient  who  is  scheduled  for  prolonged  bedrest 
are  both  at  risk  in  terms  of  skin  breakdown.  Respiratory  criteria  for 
patients  following  surgery  with  a  general  anesthetic  are  appropriate 
measures  of  effective  nursing  care  whether  the  operation  was  a  her- 
niorrhaphy, cholecystectomy,  hysterectomy,  or  a  total  hip  repair.  In 
each  case,  the  criteria  assess  the  effectiveness  of  deep  breathing  and 
coughing  exercises. 

Our  second  approach  was  to  define  nursing  problems  by  focusing  on 
the  various  systems  of  the  body,  e.g.,  respiratory,  renal,  and  cardiovas- 
cular. Continued  testing  of  the  instrument  and  development  of  the 
measurement  item  revealed  the  limited  usefulness  of  this  approach. 
For  example,  within  a  systemic  class  such  as  "gastrointestinal,"  whether 
the  patient  is  hospitalized  for  cholelithiasis  which  requires  surgical  in- 
tervention or  for  medical  therapy  alone  makes  considerable  difference 
with  respect  to  the  nursing  problems  presented  by  that  patient.  Fur- 
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thermore,  it  was  our  experience  that  when  nurses  focused  on  a  single 
system  they  tended  to  ignore  relevant  outcomes  that  involved  other 
systems.  For  example,  outcomes  related  to  problems  of  elimination 
were  missed  for  patients  with  cardio-vascular  conditions.  Systemic  clas- 
sifications do  not  define  mutually  exclusive  and  exhaustive  classes  of 
patients  which  reflect  unique  sets  of  nursing  care  problems. 

A  third  alternative,  which  avoids  these  problems,  focuses  on  the 
bio-psycho-social  functioning  of  individuals  rather  than  on  systemic 
problems  or  disease  entities.  Areas  of  healthy  functioning  of  consider- 
able interest  to  nursing  include  the  patient's  ability  to  take  in  air,  food, 
and  water;  to  distribute  these  necessities  for  life  throughout  the  body; 
and  to  carry  away  the  waste  products  of  their  utilization.  In  addition  to 
general  physical  health,  nurses  are  interested  in  those  health  behaviors 
that  afford  the  individual  a  productive,  meaningful  life.  Nurses  are 
concerned  about  the  person's  ability  to  work,  rest,  think,  play,  and  love. 

An  organizing  framework  defining  more  carefully  these  outcomes 
has  been  derived  from  earlier  work  by  Dorothea  Orem  (1971).  This 
model  is  based  upon  the  premise  that  the  special  concern  of  nursing  is, 
"man's  need  for  self-care  action  and  the  provision  and  management  of 
it  on  a  continuous  basis  in  order  to  sustain  life  and  health,  recover 
from  disease  or  injury,  and  cope  with  their  effects."  (1)  Self-care,  the 
ability  to  do  activities  of  normal  living,  consists  of  two  kinds,  universal 
and  health  deviation.  Universal  self-care  includes  all  demands  neces- 
sary for  activities  of  daily  living.  Health  deviation  self-care  demands 
arise  in  the  event  of  illness,  injury  or  disease.  For  evaluating  nursing 
care,  the  patient's  health  status  was  measured  along  dimensions  related 
to  these  requirements  for  self-care. 

Nine  categories  of  health  status  dimensions  were  developed  from 
Orem's  description  of  self-care  requirements  related  to  air;  water/fluid 
intake;  food;  elimination  of  body  wastes;  rest/activity/sleep;  solitude, 
social  interaction  and  productive  work;  protection  from  hazards;  nor- 
mality; and  health  deviation.  These  nine  aspects  of  health  provided  the 
necessary  framework  for  organizing  nursing  care  outcomes.  For  fur- 
ther specificity  in  defining  the  domain  of  nursing  care,  each  of  the 
categories  of  health  status  was  subdivided  into  four  areas:  L  Evidence 
that  the  requirement  is  met;  II.  Evidence  that  the  person  has  the  neces- 
sary knowledge  to  meet  the  requirement;  III.  Evidence  that  the  person 
has  the  necessary  skills  and  performance  abilities  to  meet  the  require- 
ment; and  IV.  Evidence  that  the  person  has  the  necessary  motivation  to 
meet  the  requirement. 

This  nine-by-four  framework  for  describing  the  scope  of  nursing's 
responsibility  was  a  valuable  heuristic  device  for  the  generation  of  pa- 
tient outcome  variables.  The  nurse  staff  selected  seven  of  the  nine 
categories  for  initial  focus  (air,  water,  food,  elimination,  rest/activity/ 
sleep,  normality  of  skin,  and  health  deviation)  and  proceeded  to  list  all 
the  variables  indicated  by  the  domains  I-IV  for  each  category.  For 
example,  variables  were  generated  that  would  demonstrate  the  degree 
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to  which  the  patient's  universal  demand  for  air  has  been  met  (domain 
I),  the  adequacy  of  knowledge  necessary  to  meet  his/her  need  for  air 
following  hospitalization  (domain  II),  and  his/her  ability  and  motiva- 
tion to  effectively  manage  self-care  related  to  getting  air  (domains  III 
and  IV).  Normal  or  optimum  levels  of  achievement  were  also  described 
for  each  variable  generated.  In  the  health  deviation  category,  ten  sub- 
sets of  dimensions  associated  with  health  deviation  and  related 
therapies  were  generated  (I.V  and  wound  observations,  knowledge  of 
health  deviation,  medication  knowledge  and  injection  performance, 
knowledge  of  diet,  fluids,  exercises,  activity  restrictions,  recommended 
rest,  special  appliances,  and  skin-wound  care).  (Table  1.) 

Validation  procedures.  Two  criteria  for  content  validity  have  been 
proposed  by  Nunnally  and  Durham  (7).  These  criteria  are  that  the 
measures  should  be  representative  of  the  domain  to  be  covered,  and 
that  the  design  of  the  measures  should  use  "sensible  methods"  of  con- 
struction. In  order  to  establish  content  validity,  three  groups  reviewed 
the  variables  and  advised  on  the  completeness  and  appropriateness  of 
their  content  with  regard  to  nursing.  These  groups  also  advised  on  the 
reasonableness  of  suggested  measurement  techniques. 

First,  the  lists  of  variables  generated  for  each  of  the  areas  of  health 
status  were  reviewed  by  all  members  of  the  research  staff.  Specific 
variables  were  judged  as  appropriate  or  inappropriate  by  collective 
opinion  as  to  whether  or  not  a  particular  variable  was  a  reflection  of 
nursing  activities.  Lists  of  appropriate  variables  were  submitted  to  the 
National  Advisory  Panel,  and  evaluated  by  the  following  two  criteria: 
(1)  the  relative  completeness  of  the  listings,  and  (2)  the  degree  to  which 
each  variable  could  be  considered  an  index  of  the  efficacy  of  nursing 
activities.  Once  variables  were  evaluated  in  this  manner,  the  National 
Advisory  Panel  was  asked  to  rank  the  variables,  using  professional 
judgment,  and  to  weight  the  relative  appropriateness  of  the  variable  as 
an  index  of  nursing  care,  to  advise  on  the  possibilities  of  devising 
measurements  for  the  variable,  and  to  determine  the  significance  of  a 
healthy  outcome  for  patients  with  respect  to  that  particular  variable. 
Following  the  meeting  with  the  Advisory  Panel,  variable  lists  were  re- 
vised. The  third  step  in  establishing  content  validity  for  each  variable 
was  to  ask  Clinical  Nurse  Specialists  practicing  in  acute  care  settings  to 
evaluate  the  completeness  of  each  of  the  variable  lists,  to  judge  the 
appropriateness  of  each  variable  as  an  index  of  nursing  care,  and  to 
rank  the  variables  using  the  same  criteria  as  those  employed  by  the 
Advisory  Panel.  The  final  lists  of  revised  variables  and  their  rankings 
are  listed  in  the  Manual,  Appendix  III,  for  each  category. 

Measurement  techniques  were  then  sought  or  developed  by  project 
staff  for  only  those  variables  which  were  valued  as  having  content  va- 
lidity and  a  high  to  moderate  sensitivity  to  nursing  care  quality.  Almost 
all  of  these  variables  fell  within  domains  I-III.  See  Table  1  for  the 
numbers  of  measurement  items  in  each  category  and  domain.  These 
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measurement  techniques  compose  the  current  instrument.  Data  collec- 
tion was  designed  to  use  the  patient  as  the  primary  source  of  data,  so 
direct  physical  observations  and  interviews  are  the  most  common  types 
of  measurement  techniques  developed.  The  measures  were  designed  so 
that  staff  nurses  could  readily  be  trained  to  do  the  observations — 
interviews,  and  so  that  the  equipment  required  would  be  available  in 
most  hospitals  or  could  be  obtained  without  great  expense. 

The  remaining  three  categories  of  health  status  variables  (solitude 
and  social  interaction,  protection  from  hazards,  and  the  remainder  of 
normality)  were  generated  as  well,  and  initial  evaluation  of  their  valid- 
ity was  begun  with  Clinical  Nurse  Specialists.  Many  of  these  variables 
are  concerned  with  measurement  of  emotional  or  affective  states.  Valid 
and  reliable  measurement  techniques  for  affective  variables,  such  as 
the  Gottschalk  Clinical  Rating  Scale  of  Anxiety  and  the  Word  Adjective 
Checklist,  have  been  documented  in  the  psychology  and  social  psychol- 
ogy literature.  These  techniques  are  appropriate  for  hospitalized  pa- 
tients who  experience  difficult  adjustments  to  their  health  problems. 
Such  measurement  techniques  can  be  incorporated  into  the  existing 
instrument  (see  (6),  Appendix  III). 
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Inter-observer 

reliability 

procedures 


Nurse  staff  members  were  used  to  establish  inter-observer  reliability 
for  each  measurement  item.  Pairs  of  nurse  staff  members  were  ran- 
domly assigned  to  measure  appropriate  items  for  selected  patients.  A 
single  reliability  trial  consisted  of  a  pair  of  observers  who  approached  a 
patient  jointly.  One  randomly  chosen  member  of  the  pair  directed  the 
interview  and  performed  the  tests  indicated  while  the  other  member 
observed.  For  physical  examination  items,  each  observer  performed 
the  necessary  observations  and  recorded  the  results.  In  all  trials  both 
observers  recorded  their  observations  and  scored  patient  responses  in- 
dependently. These  records  were  returned  to  a  third  party  who  com- 
pared the  records  and  reported  the  results  of  each  trial  for  each  vari- 
able as  an  instance  of  agreement  or  disagreement. 

The  criteria  for  determining  that  the  measurement  technique  for  a 
particular  variable  was  invariant  over  observers  were:  (1)  a  sample  re- 
liability index  of  at  least  0.80;  (2)  an  alpha  error  of  0.05;  and  (3)  the 
lower  bound  of  the  95  percent  confidence  interval  to  exclude  the  relia- 
bility index  of  0.60. 

Decisions  on  variables  that  fall  below  the  standard.  A  general 
model  for  decision-making  on  the  nature  of  subsequent  work  required 
when  a  reliability  test  for  a  variable  failed  appears  in  Figure  1.  Essen- 
tially, the  project  staff  would  review  the  data  on  a  variable  to  deter- 
mine if  there  was  a  correct  method  of  measuring  the  variable.  If  so, 
information  from  individual  observers  was  reviewed  to  determine 
which  persons  may  have  measured  variables  incorrectly.  It  appeared 
reasonable  to  question  the  adequacy  of  the  training  of  observers  as  the 
first  possible  source  of  unreliability.  First  remedies  for  unreliability 
were  to  reopen  discussions  on  particular  measures  to  retrain  the  obser- 
vers. 

When  it  was  shown  that  training  was  not  the  problem,  test  results 
that  failed  to  show  inter-observer  reliability  argued  strongly  for  the 
necessity  of  revising  the  measuring  technique  for  a  particular  variable. 
The  last  node  of  the  decision-tree  then  became  important.  Given  the 
principle  that  one  never  has  resources  sufficient  to  accomplish  all  tasks 
to  be  done,  we  sought  to  maximize  the  utility  of  the  total  instrument  by 
working  f  irst  on  those  variables  that  described  outcomes  for  the  great- 
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est  number  of  patients.  We  decided  to  work  only  on  those  variables 
that  were  appropriate  for  at  least  20  percent  of  the  patients  covered  by 
the  instrument.  The  remainder  of  the  project's  resources  went  into 
conducting  the  reliability  and  validity  tests  of  those  other  variables. 
Variables  that  failed  the  reliability  test  and  did  not  meet  the  applicabil- 
ity criterion  stated  above  were  set  aside. 
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Scope 

of  the  current 
instrument 


Validation  procedures  utilizing  the  project  staff,  the  National  Advisory 
Panel  and  the  Clinical  Nurse  Specialists  have  resulted  in  an  instrument 
containing  539  measurement  items.  Of  these  measurements  328  are 
related  to  the  universal  demands,  for  ait  (70),  food  (22),  elimination 
(73),  water  (29),  rest/activity/sleep  (110),  solitude  and  social  interaction 
(1),  and  normality  (23).  The  health  deviation  section  of  the  instrument 
(211  measurements)  describes  intravenous  and  wound  observations 
(28),  patients'  knowledge  and  self-care  abilities  relative  to  their  health 
problems  (13),  their  medications  (61),  therapeutic  diet  (23),  and  fluid 
intake  (11),  prescribed  rehabilitative  exercises  (17),  restrictions  on 
physical  activity  (11),  skin  and  wound  care  (19),  the  use  of  special 
appliances  (15)  and  recommended  rest  (13).  See  (6),  Appendix  II  for 
the  instrument. 

Domains  I-III  are  quite  evenly  represented  over  the  instrument. 
There  are  198  variables  in  Domain  I  (Universal,  170  variables;  Health 
Deviation,  28  variables).  Doman  II  is  represented  by  183  variables  (30 
in  Universal  categories,  153  in  Health  Deviation).  Doman  III  includes 
155  variables  (Universal,  125;  Health  Deviation,  30).  Domain  IV,  which 
requires  affective  measures,  currently  has  3  variables  (Table  1). 

Table  2  gives  an  overview  of  the  types  of  items  in  the  instrument.  Of 
the  instrument  items,  469  are  appropriate  as  quality  measures:  414  of 
these  were  tested  for  reliability;  the  other  55  items  were  revised  or  de- 
veloped after  reliability  testing  was  completed  and  therefore  require 
further  tests.  The  remaining  70  informational  items  aid  interpretation 
of  the  quality  measures  and  do  not  require  reliability  tests. 

Finally,  we  have  drafted  health  status  dimensions  and  levels  of 
achievement,  but  not  measurement  techniques,  in  the  areas  of 
solitude-social  interaction,  protection  from  hazards,  and  a  large  por- 
tion of  normality.  Future  research  is  planned  to  search  out  appropriate 
measures  in  the  literature  for  these  health  status  dimensions. 
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Validity 


Content  validity  has  been  established  for  all  539  measurements  on  the 
basis  of  two  criteria  (7):  that  items  in  the  instrument  be  representative 
of  the  domain  to  be  covered  by  that  instrument,  and  that  the  design  of 
the  measures  themselves  utilize  "sensible  methods"  of  construction. 
Variables  identified  within  the  instrument  were  chosen  because  they 
represented  physiological,  psycho-motor,  cognitive,  or  affective  com- 
ponents of  each  of  the  areas  of  concern  to  nursing,  whether  related  to 
universal  demands  or  to  demands  arising  from  health  deviation. 
Opinions  of  the  quality  staff  about  the  representativeness  of  each  vari- 
able as  an  indicant  of  the  quality  of  nursing  care  were  verified  by  two 
additional  groups  of  experts,  the  practicing  clinical  specialists  and  the 
National  Advisory  Panel.  The  "reasonableness"  of  suggested  measur- 
ing techniques  was  also  subjected  to  critical  confirmation  by  the  two 
groups. 

Validation — case  examples.  As  reliability  testing  was  being  done, 
anecdotal  notes  about  the  process  were  recorded  and  cases  of  special 
interest  were  described.  Several  cases  demonstrate  some  degree  of  con- 
struct validity  for  several  of  the  measures. 

Two  such  measures  were  in  the  component  "air."  The  performance 
test  for  the  health  status  dimension  "adaptation  to  activity"  is  a  meas- 
ure of  the  patient's  ability  to  adjust  his  performance  of  activities  of 
daily  living  to  his  needs  for  oxygen.  The  variables  measured  are  de- 
grees of  shortness  of  breath,  recovery  heart  rates,  and  recovery  res- 
piratory rates.  If  the  patient  is  satisfactorily  adjusting  his  activity  to  his 
needs,  his  heart  and  respiratory  rates  should  return  to  resting  levels 
within  90  seconds  after  completing  the  activity.  Most  patients  tested  for 
this  item  had  respiratory  disease  causing  shortness  of  breath  on  exer- 
tion. Most  of  the  patients  achieved  the  normal  recovery  levels  within  90 
seconds,  taking  as  many  rest  stops  as  they  needed.  In  one  case,  how- 
ever, a  patient,  who  had  congestive  heart  failure  and  asthma,  did  not 
rest  as  needed,  but  in  fact  hurried  through  her  assigned  activity, 
bathing  at  the  bedside,  in  order  to  get  ready  for  some  visitors.  She 
notably  failed  to  meet  the  recovery  levels  at  the  90-second  limit.  Fez 
this  patient,  the  measures  were  indeed  sensitive  to  the  patient's  adjust- 
ment of  her  activity  levels. 

A  second  measure  in  the  "air"  component  deals  with  the  patient's 
emotional  adjustment  to  chronic  cough  and  sputum  production.  The 

9 


questions  focus  on  the  patient's  solutions  to  any  embarrassment  or 
value  conflict  caused  by  coughing  and  expectoration.  Overall,  re- 
sponses indicated  that  the  respiratory  disease  patients  interviewed  had 
either  adjusted  to  the  point  that  they  felt  no  embarrassment  or  had 
developed  adequate  solutions.  In  one  case,  however,  the  patient,  who 
had  restrictive  lung  disease  and  bronchitis,  expressed  solutions  that  in- 
cluded extensive  withdrawal  from  social  contacts  and  outside  activities. 
She  viewed  these  solutions  as  the  only  course  open;  yet  her  expression 
of  frustration  and  anger  indicated  her  lack  of  satisfaction  with  these 
solutions.  Thus  the  questions  have  been  found  to  elicit  both  positive 
and  negative  feeling  states  related  to  adaptation  to  a  disease  process. 

Several  cases  demonstrate  construct  validity  for  performance  meas- 
ures in  the  "food"  component.  The  measures  test  the  patient's  ability  to 
perform  activities  of  eating,  such  as  mashing,  cutting  food,  pouring 
and  lifting  and  opening  containers.  For  a  patient  with  a  17-year  history 
of  rheumatoid  arthritis,  the  measures  demonstrated  inability  to  mash 
food  and  some  difficulty  in  cutting.  The  patient  was  suffering  a 
"flare-up"  of  his  arthritis  and  stated  that  he  usually  could  manage 
these  activities  at  home.  When  asked  for  his  opinion  as  to  whether  the 
tests  would  demonstrate  at  discharge  any  problems  he  would  have  at 
home,  he  stated  "yes,  it  certainly  would."  For  another  patient,  with  a 
diagnosis  of  multiple  sclerosis,  the  measures  demonstrated  that  the 
muscle  weakness  in  his  hands  and  arms,  which  had  prevented  him 
from  lifting  a  quart  container  of  liquid  at  home,  had  resolved  to  the 
extent  that  he  could  lift  it  at  the  time  of  discharge.  He,  too,  stated  that 
these  tests  were  appropriate  for  evaluating  his  abilities  to  eat  at  home. 

The  observers  have  also  noted  that  the  range  of  motion,  muscle 
strength,  muscle  endurance,  and  coordination  measures  in  the  Rest/ 
Activity/Sleep  component  have  distinguished  problems  with  motion 
from  those  of  strength,  and/or  coordination.  The  rheumatoid  arthrit- 
ic's  coordination  and  strength  tests  were  fairly  normal  despite  a  notably 
decreased  range  of  motion.  This  was  the  same  patient  who  had  diffi- 
culty mashing  and  cutting  food.  Thus  the  tests  indicated  that  it  was 
primarily  his  more  limited  range  of  motion  which  was  affecting  his 
eating  activity.  Another  patient  demonstrated  good  strength  on  the 
muscle  strength-endurance  tests  except  in  his  hands  and  feet.  The 
range  of  motion  tests  also  showed  some  minor  restrictions  in  hands  and 
feet.  After  the  interview,  the  patient  told  the  observers  that  he  did  have 
some  restricted  motion  in  his  hands  and  feet  due  to  a  past  history  of 
myasthenia  gravis,  information  that  was  not  pertinent  to  his  present 
diagnosis  and  did  not  appear  on  the  face  sheet  of  the  patient  record. 
The  loss  of  motion  and  strength  was  not  large  enough  to  significant lv 
affect  his  daily  activities.  Thus  the  measures  were  sensitive  to  a 
relatively  minor  loss  of  function.  Strength  and  active-passive  range  of 
motion  tests  corroborated,  suggesting  that  the  loss  of  strength  had 
apparent  1\  allowed  some  minor  contractures  to  develop. 

Because  all  of  these  observations  were  done  for  purposes  of  estab- 
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lishing  reliability,  the  measures  were  not  necessarily  obtained  at  the 
times  most  appropriate  for  evaluation  of  nursing  care.  They  do,  how- 
ever, demonstrate  that  the  measures  are  sensitive  to  differences  in  pa- 
tients' health  status.  If  obtained  at  appropriate  times,  the  tests 
could  validly  indicate  (for  groups  of  similar  patients)  strengths  and  de- 
ficiencies in  the  care;  that  is,  whether  nurses  followed  procedures  to 
prevent  contractures  and  to  maintain  muscle  strength,  adequately  pre- 
pared patients  for  performing  daily  activities  at  home,  or  assisted  pa- 
tients to  resolve  problems  related  to  adjustment  to  disease. 
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Reliability 


Of  the  414  quality  measures  in  the  instrument  (Table  2),  109  or  26 
percent  attained  the  stringent  criteria  set  for  reliability.  That  is,  for 
those  109  measures  the  pairs  of  nurse  observers  showed  a  0.80  index  of 
agreement  which  was  statistically  significant  at  the  0.05  level;  further, 
the  confidence  limits  for  that  coefficient  excluded  the  0.60  index  at  its 
lower  bound.  An  additional  31  measures  were  shown  to  meet  at  least 
two  of  the  criteria;  the  index  of  agreement  was  0.80  or  better  and  was 
statistically  significant  at  the  0.05  level.  The  criterion  with  respect  to 
confidence  bounds  was  not  attained  for  140  measures  because  agree- 
ment was  absolute  (r  =  1.00)  and,  thus,  no  bounds  were  determinable. 
These  280  measures  which  have  shown  reliability  cut  across  the  scope 
of  the  instrument.  For  most  sections  of  the  instrument,  at  least  some  of 
the  measures  are  both  valid  and  reliable.  The  summary  of  these  find- 
ings are  tabulated  in  Table  3.  Findings  for  the  total  item  set  are  tabu- 
lated in  (6),  Appendix  I,  Reliability  Test  Results. 

One  hundred  and  eight  of  the  measures  need  further  testing  because 
inter-observer  reliability  was  not  established  for  these  measures. 
Another  problem  arose  for  the  remaining  26  measures.  These  par- 
ticular measures  appear  to  be  appropriate  only  for  a  limited  subset  of 
the  population,  hospitalized  adult  patients.  We  were  unable  to  find  suf- 
ficient persons  for  whom  such  measures  could  be  used,  despite  persis- 
tent attempts.  These  measures  would  appear  to  have  limited  utility  for 
assessment  of  the  quality  of  nursing  care,  since  few  patients  would  be 
available  to  determine  an  index  of  quality. 
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Manual 


In  order  for  others  to  utilize  the  items  appropriately  and  to  measure 
each  item  reliably,  it  was  necessary  to  prepare  a  manual  with  detailed 
instructions  on  the  use  of  the  instrument. 

The  manual  (Appendix  III)  (6)  supplies  information  about  the  pa- 
tient observations,  interviews,  scoring  procedures,  and  other  guidelines 
for  recording  accurate  information  about  the  health  status  of  a  par- 
ticular patient.  It  also  includes  guidelines  for  deciding  whether  or  not  a 
particular  measurement  item  is  appropriate  for  patients  in  various 
diagnostic  categories.  The  manual  is  intended  to  be  useful  both  for 
training  of  nurse  observers  and  also  as  a  reference  source  during  ac- 
tual data  collection  periods.  An  overview  of  its  development  is  de- 
scribed below. 


Development  Of  manual.  As  each  measurement  technique  was  de- 
signed, instructions  to  the  observer/interviewer  on  how  to  perform  the 
measure  were  developed.  To  aid  evaluation  of  the  observations  and 
responses,  norms  or  standards  documented  in  the  literature  were  in- 
corporated into  the  manual.  In  those  cases  where  documented  "cor- 
rect" responses  were  not  available,  examples  of  acceptable  responses 
were  listed.  Three  sections  of  the  manual  have  fairly  extensive  lists  of 
"correct"  responses:  Health  Deviation  Knowledge,  Medications,  and 
Special  Appliances.  For  Health  Deviation  Knowledge,  correct  re- 
sponses were  detailed  for  H-ICDA  diagnostic  categories  frequently  oc- 
curring in  several  hospitals  of  varying  size,  character,  and  location.  For 
the  Special  Appliances  and  Medication  components,  correct  response 
lists  were  prepared  for  commonly  prescribed  medications  and 
appliances. 

After  reliability  testing  was  completed,  the  manual  was  revised  and 
improved,  incorporating  knowledge  gained  from  the  use  of  the  in- 
strument in  data  collection.  It  currently  includes  an  introduction  to  the 
instrument  and  its  use;  instructions  to  data  collectors;  guidelines  for 
reliability  testing;  equipment  needed;  detailed  descriptions  of  each  in- 
strument component,  including  the  complete  lists  of  health  status  di- 
mensions generated;  a  guide  for  selection  and  matching  of  patients 
and  appropriate  measures;  a  discussion  of  interviewing  techniques; 
and  procedures  for  scoring  responses. 
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Patient  selection  guide.  The  patient  selection  guide  (Appendix  B) 
designates  the  appropriate  health  status  dimensions  to  measure  for  90 
frequently  occuring  H-ICDA  diagnostic  categories.  It  was  generated 
with  extensive  input  by  clinical  nurse  specialists  in  order  to  determine 
which  items  would  be  appropriate  measures  of  nursing  care  for  specific 
patient  groups.  It  was  used  extensively  during  reliability  testing  and  is 
viewed  as  a  substantial  beginning  effort  to  match  measures  to  patients 
in  a  valid  way  to  evaluate  quality  of  nursing  care.  By  linking  H-ICDA 
code  categories  to  appropriate  sets  of  measurement  items  in  the  in- 
strument, the  matrix  capitalizes  on  the  information  system  already 
available  in  hospital  settings.  The  guide  can  be  used  in  various  geo- 
graphical locations  and  is  amenable  to  computerization.  It  is  feasible 
that  both  patient  selection  and  production  of  appropriate  data  collec- 
tion materials  could  be  computerized  as  linked  operations  using  this 
matrix. 
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Summary 


The  work  of  the  project  staff  has  yielded  an  instrument  for  measuring 
patients'  health  status  along  a  number  of  dimensions  under  the  influ- 
ence of  nursing  care.  The  project  staff  considers  all  measures  where 
the  sample  inter-observer  reliability  index  was  at  least  0.80  (a  <  .05)  to 
be  reliable.  Failure  to  achieve  the  confidence  limit  criterion  was  a  mat- 
ter of  insufficient  trails  to  stabilize  the  reliability  index  above  the  0.60 
lower  confidence  bound.  Therefore,  we  posit  an  instrument  of  348 
valid  and  reliable  measures  for  general  use  in  assessing  the  quality  of 
nursing  care  based  on  observations  of  patients'  health  status.  These 
measures  include  280  reliable  quality  measures  and  68  informational 
data  items  (Table  3).  For  the  universal  categories  they  total  248  items, 
and  include  air  (53),  water  (23),  food  (15),  elimination  (64),  rest/ 
activity/sleep  (77),  solitude  and  social  interaction  (1)  and  normality 
(15).  The  remainder  of  the  instrument  (100  measures)  relates  to  de- 
mands arising  from  health  deviations.  Included  in  this  section  of  the 
instrument  are  observations  of  I.V.  sites  and  wounds  (23),  measures  of 
health  deviation  knowledge  (5),  medication  knowledge  (18),  injection 
performance  (14),  therapeutic  diet  knowledge  (7),  knowledge  of 
therapeutic  fluids  (6),  knowledge  of  physical  activity  restrictions  (3), 
use  of  special  appliances  (10),  and  care  of  the  skin  and  wounds  (4). 

The  measures  focus  on  the  patient  as  the  primary  source  of  data, 
and  thus  require  skills  in  physical  observation  and  interviewing  tech- 
niques. This  necessitates  selecting  or  training  appropriately  skilled 
nurse-observers.  Needed  equipment  is  readily  available  and  inexpen- 
sive. 

The  instrument  as  designed  is  most  appropriate  for  research  on  al- 
ternative models  for  the  delivery  of  nursing  care  services.  Subsections 
of  the  instrument  may  be  used  to  assess  the  relative  efficacy  of  alterna- 
tive nursing  care  interventions  for  particular  patient  problems. 
Selected  items  could  be  used  in  quality  assurance  programs  at  the  dis- 
cretion of  the  users.  For  any  of  these  applications,  a  manual  accom- 
panies the  instrument  in  order  that  others  may  use  each  of  the  meas- 
ures reliably  and  effectively. 
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Definitions 


Health — a  state  of  complete  physical,  mental  and  social  well-being  and 
not  merely  the  absence  of  disease  or  infirmity.5 

Health  status — condition  or  position  with  respect  to  health. 

Health  status  may  be  viewed  on  a  continuum,  ranging  from  optimum 
health  to  death,  and  defined  by  the  life  cycle  or  by  recuperative  proc- 
esses after  specific  illnesses  or  therapies/interventions. 

Health  status  dimension — an  aspect  of  health  status;  a  continuum 
along  which  various  levels  of  achievement  are  possible  and  measure- 
able. 

Specified  levels  of  achievement  along  particular  health  status  dimen- 
sions may  be  used  as  criteria  for  evaluating  the  nursing  care  of  selected 
patient  populations. 

Norm — such  a  standard  of  achievement  as  represented  by  the  median 
or  average  achievement  of  a  large  group.8  Normal  is  the  adjectival 
form. 

Optimum  level  of  achievement — that  level  of  achievement  on  a  health 
status  dimension,  of  a  person  or  group  with  a  health  deviation,  at 
which  the  person(s)  attain  the  closest  possible  approximation  to  health, 
assuming  no  other  impairment  limits  the  person(s)  ability  to  achieve  on 
the  dimension. 

Outcome — result,  consequence.  Synonym:  effect.6 

Assessment  of  outcomes  is  the  evaluation  of  end  results  in  terms  of 
health  and  satisfaction.7 

Reliability — the  accuracy  or  consistency  of  a  measuring  instrument. 
The  consistency  with  which  an  instrument  yeilds  similar  repeated 
measurements  of  the  same  phenomenon. 

Standard — a  professionally  developed  or  agreed  upon  level  of  attain- 
ment which  is  regarded  as  a  measure  of  adequacy. 

Validity — the  degree  to  which  a  measurement  has  produced  a  "true" 
representation  of  the  phenomenon  being  measured  without  the  influ- 
ence of  other  phenomena.8 

Variable — a  property  that  may  have  a  number  of  different  values. 
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Figure  1.  Decision-tree  for  action  following  failure  of  reliability  test 

for  a  variable 
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Tables 


Table  1:  Number  of  measurement  items  in  health  status  categories 

and  domains 


Categories 

Total  Measurement  Items  in  Domains 

/ 

II 

///  IV 

Requirements 

K  YlOwl€(lg€ 

Performance  Motivation 

Total 

Total 

Met 

U niversal  Dimensions: 

1.  Air 

41 

6 

20  3 

70 

2.  Water 

19 

5 

5 

29 

3.  Food 

3 

3 

16 

22 

4.  Elimination 

16 

16 

41 

73 

5.  Rest/Activitv/Sleep 

67 

43 

110 

6.  Solitude  & 

Social  Interaction 

1 

1 

7.  Protection  from  hazards 

8.  Normality 

23 

23 

Subtotal 

170 

30 

125  3 

328 

Health  Deviation  Dimensions: 

9.  Health  Deviation  I  I.V. 

8c  Wound  Observation 

28 

28 

10.  Health  Deviation  II 

13 

13 

11.  Medications 

31 

30 

61 

12.  Therapeutic  Diet 

23 

(Injections) 

23 

13.  Therapeutic  Fluids 

11 

11 

14.  Recommended  Exercises 

17 

17 

15.  Physical  Activity 

Restrictions 

1 1 

11 

16.  Recommended  Rest 

13 

13 

17.  Special  Appliances 

15 

15 

18.  Skin-Wound  Care 

19 

19 

Subtotal 

28 

153 

30 

211 

TOTAL 

198 

183 

155  3 

539 
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Table  2:  Type  and  status  of  measurement  items  in  current  instru- 
ment 


Qut 

ility  measurement  items 

Informational 

items 

(  A?  e]in  hi  litv 

(  Ac iiuuiiny 

Tested  for 

Revisions 

tests  unnec- 

Total 

Component 

reliability 

new  items 

Total 

essary) 

items 

U  niversal 

1.  Air 

65 

2 

67 

3 

70 

2.  Water 

25 

2 

27 

2 

29 

3.  Food 

15 

4 

19 

3 

22 

4.  Elimination 

44 

45 

28 

73 

5.  Rest/Activity/Sleep 

83 

4 

87 

23 

110 

6.  Solitude  & 

Social  interaction 

1 

1 

7.  Protection  from  Hazards 

8.  Normality 

18 

5 

23 

23 

Health  Deviation 

9.  Health  Deviation  I  I.V. 

&  Wound  Observation 

23 

5 

28 

28 

10.  Health  Deviation  II 

12 

1 

13 

13 

11.  Medications 

46 

12 

58 

3 

61 

12.  Therapeutic  Diet 

20 

2 

22 

1 

23 

13.  Therapeutic  Fluids 

7 

1 

8 

3 

11 

14.  Recommended  Exercises 

13 

3 

16 

17 

15.  Physical  Activity 

Restrictions 

6 

4 

10 

l 

11 

16.  Recommended  Rest 

10 

2 

12 

l 

13 

17.  Special  Appliances 

11 

4 

15 

15 

18.  Skin-Wound  Care 

15 

3 

18 

l 

19 

Total 

414 

55 

469 

70 

539 

Percent  of 

total  items 

77% 

10% 

87% 

13% 
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Table  3:  Summary  of  reliability  testing  on  the  instrument 


Number  of  Measurement  Items  Meeting  Criteria 


Index  =  .80 

Insufi- 

a  <  .05 

Index  =  .80 

Index = 

Index 

ficient 

Component 

Lower  bound  <.60 

a<.05 

1.00 

<  .80 

patients 

Total 

Universal 

1.  Air 

33 

6 

1  1 

1 5 

65 

2.  Water 

3 

1 

12 

3 

J 

25 

3.  Food 

2 

1 

9 

3 

15 

4.  Elimination 

7 

29 

8 

44 

5.  Rest/Activitv/Sleep 

13 

9 

32 

29 

83 

6.  Solitude  8c  Social 

Interation 

1 

1 

7.  Protection  from 

Hazards 

8.  Normality 

11 

4 

3 

18 

Health  Deviation 

9.  Health  Deviation  I 

I.V.  &  Wound 

Observation 

13 

5 

5 

23 

10.  Health  Deviation  II 

3 

2 

7 

12 

11.  Medications 

11 

2 

16 

17 

46 

12.  Therapeutic  Diet 

2 

2 

2 

13 

1 

20 

13.  Therapeutic  Fluids 

1 

2 

3 

1 

7 

14.  Recommended 

Exercises 

13 

13 

15.  Physical  Activity 

Restrictions 

2 

4 

6 

16.  Recommended  Rest 

10 

10 

17.  Special  Appliances 

6 

4 

i 

11 

18.  Skin-wound  Care 

2 

1  1 

2 

15 

Total  Items 

109 

31 

140 

108 

26 

414 

Percent  of 

tested  items 

26% 

8% 

34% 

26% 

6% 

Reliable 

Not  reliable 

280  (68%) 

134  (32%) 
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